perf(build): replace specs glob with fs reads#2959
Conversation
The 11 import.meta.glob calls in specs.ts expanded into 58k lazy import statements, producing 49,123 server chunks (500 MB) and pushing peak build RSS to 7 GB. The .md and .json files were being treated as JS modules with full transform/sourcemap/chunk overhead, when they are just static text. Resolve @appwrite.io/specs at runtime via createRequire and load examples + OpenAPI specs with fs.readFile. Move the package from devDependencies to dependencies so it survives bun install --production and ships to the final image. Build wall: 192s -> 113s. Peak RSS: 7.0 GB -> 5.6 GB. Server chunks: 49,123 -> 3,194. Final image net ~125 MB smaller (build/server drops 421 MB, node_modules adds 296 MB). Drops vite-plugin-dynamic-import which is no longer needed.
Appwrite WebsiteProject ID: Website (appwrite/website)Project ID: Tip Schedule functions to run as often as every minute with cron expressions |
Greptile SummaryThis PR eliminates the 11
Confidence Score: 4/5The build optimisation is sound and the key correctness concerns (version validation, path traversal, parallel example reads, JSON caching) are all addressed in the new code. The rewrite is structurally correct and the test plan is comprehensive. The open item from a prior review thread (cache key should be version|mode not version|platform) is still unaddressed, so on a busy SSR server the same multi-MB JSON document can accumulate one copy per server-side platform variant. No new blocking correctness issues were found in this pass. src/routes/docs/references/[version]/[platform]/[service]/specs.ts — specifically the apiCache key at line 414 and the locateSpecsRoot fallback candidate paths, which assume a fixed SvelteKit output depth. Important Files Changed
Reviews (5): Last reviewed commit: "Merge branch 'main' into website-optimiz..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
…RL params - Cache parsed OpenAPI documents in a module-scope Map keyed by version|platform. Restores the implicit caching the old import.meta.glob path got from Node's ESM module cache, avoids re-parsing multi-MB JSON on every request. - Replace the serial loadExample await loop with Promise.all over the prepared method list. Cuts I/O latency on services with many endpoints. - Validate version and platform against allowlists from references.ts before any path.join. URL segments shouldn't reach a filesystem read unguarded; a lone .. segment could escape examples/ otherwise.
getApi only interpolates a derived literal mode (server/client/console)
into the JSON path, never platform itself, so the platform allowlist
there was rejecting valid internal callers (model-markdown.ts and
models/[model]/+page.server.ts hardcode 'console-web' which isn't in
the public Platform enum).
Move the guard to getService where the raw platform value is actually
interpolated into examples/${version}/${platform}/... — that's the
real filesystem boundary. URL paths reaching getService still go
through the [platform] route segment whose values come from the
Platform enum, so the check stays correct there.
Fixes 69 failing redirect tests for /docs/references/cloud/models/*.
| const cacheKey = `${version}|${platform}`; | ||
| const cached = apiCache.get(cacheKey); | ||
| if (cached) { | ||
| return cached; | ||
| } | ||
|
|
||
| const isClient = platform.startsWith('client-'); | ||
| const mode = platform.startsWith('server-') ? 'server' : isClient ? 'client' : 'console'; | ||
| const filename = `open-api3-${version}-${mode}.json`; |
There was a problem hiding this comment.
apiCache key includes the full platform string (e.g. server-nodejs, server-python), but the JSON file actually read is determined by mode — one of three fixed literals (server, client, console). Every server-side platform for the same version maps to the same open-api3-${version}-server.json file, so each distinct platform string creates a separate cache entry that parses and stores an identical multi-MB document. On a live SSR server where different users request server-nodejs, server-python, server-php, etc. concurrently, the same JSON is parsed and allocated once per platform variant — partially undoing the memory benefit of the cache. The key should be ${version}|${mode} so all server platforms share one entry.
| const cacheKey = `${version}|${platform}`; | |
| const cached = apiCache.get(cacheKey); | |
| if (cached) { | |
| return cached; | |
| } | |
| const isClient = platform.startsWith('client-'); | |
| const mode = platform.startsWith('server-') ? 'server' : isClient ? 'client' : 'console'; | |
| const filename = `open-api3-${version}-${mode}.json`; | |
| const isClient = platform.startsWith('client-'); | |
| const mode = platform.startsWith('server-') ? 'server' : isClient ? 'client' : 'console'; | |
| const cacheKey = `${version}|${mode}`; | |
| const cached = apiCache.get(cacheKey); | |
| if (cached) { | |
| return cached; | |
| } | |
| const filename = `open-api3-${version}-${mode}.json`; |
# Conflicts: # bun.lock # package.json
2 participants
Summary
The 11
import.meta.globcalls inspecs.ts(one per Appwrite version) expanded into ~58k lazyimport()statements, producing 49,123 server chunks (500 MB) and pushing peak build RSS to 7 GB. The matched.mdand.jsonfiles are static text — there's no reason to route them through Vite's module graph.This PR resolves
@appwrite.io/specsat runtime viacreateRequireand reads examples + OpenAPI specs withfs.readFile. The package moves fromdevDependenciestodependenciesso it survivesbun install --productionand ships to the final image.vite-plugin-dynamic-importis removed (no longer needed).Build impact
build/server/sizeTest plan
/docs/references/cloud/server-nodejs/databasesrenders code examples (Node.js)/docs/references/1.5.x/server-python/storagerenders (Python, older version)/docs/references/1.7.x/client-android-kotlin/accountrenders (exercises the Android special path inloadExample)